The present invention relates to a multimedia communication system. More specifically, the present invention relates to an audio data gateway, an audio data relaying method, and a multiplexing format for the relaying of audio data.
There has been rapid growth of the Internet, which is based on TCP/IP. The WWW (World Wide Web), a global data communication system, has provided further acceleration to the widespread use of the Internet, and the transfer of multimedia data, including audio and image data, over the Internet has become commonplace.
Against this backdrop, communication systems referred to as Internet telephony are emerging to provide communication similar to telephones by connecting communication terminals to the Internet for sending and receiving real-time audio data. There have been increased efforts recently to unify communications over the Internet. In particular, there has been significant interest in Internet telephones where a phone call is made by using a standard telephone to access an access point on the Internet. The phone call goes from there to an access point that is near the person being called. From this access point, the call goes to the public telephone network. For example, if access points are set up at both Tokyo and Osaka, the phone charge for a call from Tokyo to Osaka would be made up of the phone charge from the user to the Tokyo access point and the phone charge from the Osaka access point to the person being called. This would be cheaper than using the public telephone network to make the call from Tokyo to Osaka. This type of service is expected to grow in the future. Internet telephony is presented in xe2x80x9cCheap calls using the Internet: Three minutes from Tokyo to Osaka for 55 yenxe2x80x9d (Nikkei Communication, Feb. 3, 1997), and xe2x80x9cInternet telephony and video phonesxe2x80x9d (PCWAVE, pg. 81, March 1997).
With the conventional communication systems described in the above references, if a single audio data gateway is used for multiple lines, gateway operations are performed by generating separate audio streams for each line. To send the audio data over the Internet, these communication systems must add to the audio data their own headers or headers as set forward by standards committees, as well as header information from the different communication protocol layers. Packets must then be formed from this data. The audio data itself is split up into very short packets, e.g., less than 100 ms. Thus, the size of the additional data becomes many times the size of the audio data. Multiple audio streams relayed to a particular target gateway will take up excessive bandwidth despite the fact that much of the added data in the streams is similar. As an example, we will look at packets based on the UDP/IP communication protocol widely used over the Internet and the RTP real-time communication protocol that has been proposed by the IETF (Internet Engineering Task Force), an industry standards association. In these packets, 30 ms of audio data (using G.723.1 encoding at 5.3 kbps) would take 20 bytes, the RTP header would take 2 bytes, the UDP header would take 8 bytes and the IP header would take 20 bytes. Thus, 40 bytes of header data is twice the size of the audio data. If different applications add additional header data to this, the bandwidth taken up by the header increases.
On the input side, analog audio data is converted to digital audio data, and digital audio data is converted to analog audio data on the output side. Since conversion between analog and digital data is performed in hardware, input/output operations between the hardware and the software must be performed, and this takes up CPU power. In general, input/output operations between hardware and software take up a few milliseconds. When generating packets at 20 ms intervals, input/output operations take place 50 times a second per line. Since each operation takes a few milliseconds, the processing time required for 50 input/output operations is at least 100 ms. If there are multiple active lines, the input/output operations will be more frequent. This is a significant factor in audio communications, which must be performed in real time.
An object of the present invention is to provide a data gateway and a data relaying method that can reduce the bandwidth used by multiple streams having the same destination by multiplexing data that can be shared such as header data.
Another object of the present invention is to provide a data gateway and a data relaying method that can reduce the number of input/output operations between hardware and software that require CPU power for streams that can be multiplexed.
Yet another object of the present invention is to provide a data gateway and a data relaying method where, if multiple streams have the same destination gateway and can be multiplexed, a gateway that handles multiple lines can negotiate with a destination gateway to determine whether to perform multiplexing and whether the destination gateway supports multiplexing.
Yet another object of the present invention is to provide a multimedia communication system that uses the audio data gateway described above.
Yet another object of the present invention is to provide a control unit that uses the audio data gateway described above and that performs administrative tasks for the audio data gateway device.
In order to achieve the objects described above, the present invention provides an audio data gateway that includes: means for receiving an Internet relay request and a number called in a call from a telephone via the Internet, selecting a destination audio data gateway using the number called, and performing relay operations with the selected destination audio data gateway; and means for multiplexing relay audio data for streams based on multiple relay requests having the same relay destination, i.e. streams bound for the same destination and having common data so that the relay audio data in these streams (the non-shared data of these streams) are multiplexed into a single relay stream.
In this communication system, a plurality of relay streams are multiplexed into a single relay stream. This makes it possible to conserve more bandwidth compared to relaying each relay stream independently.
Also, the hardware for performing analog/digital conversion of audio data includes means for performing batch transfer to higher-level software of multiple sets of converted audio data that are bound for the same relay destination.
Since this communication system performs input and output operations between hardware and software all at once, the number of input and output operations, which require CPU power, can be reduced and more lines can be handled.
In the audio data gateway according to the present invention, the packet format for multiplexed streams includes at least an identifier to indicate whether the packet is a multiplexed stream, as well as means for negotiating with the destination on whether multiplexing is to be performed when a new stream is created. This audio data relaying system also includes a control unit for performing administrative operations on the audio data gateway. The control unit includes: means for monitoring, which monitors proper operation of the audio data gateways; relay logs (the phone numbers of users who made calls, the phone numbers called, the times at which calls were begun, the times at which calls were terminated, and the like) for each audio data gateway; means for periodically collecting relay packet loss and relay packet delays; and means for sending scheduling information for each audio data gateway.
In the data relaying method according to the present invention, a request for a call and a telephone number to call are received from a telephone. Using the number being called, the gateway closest to the destination is found and the address of the resulting destination gateway is obtained. A connection request is established using the address of this gateway. Analog audio data from the telephone is converted into digital data. The converted digital data is placed in packets and relayed to the destination gateway. The relaying to the destination gateway is performed by: temporarily storing the converted digital data; when multiplex timing is reached, the stored digital data is retrieved and streams bound for the same destination gateway are multiplexed into a single packet.
The present invention also provides a method for connecting points on a public telephone network using the Internet and relaying data streams using IP packets. During the relaying of a plurality of audio data streams, if streams bound for the same destination IP address are created, negotiations are performed with the destination audio gateway to determine if the streams should be relayed in a multiplexed manner. If multiplexing is to be performed as a result of the negotiation, multiplexed data is generated and sending and receiving of data takes place.
In the data gateway of the present invention, if relay data is being sent and received using UDP and a new connection request is generated, the connection establish module includes means for assigning UDP ports assigning a separate UDP port to the data stream distinct from those of the existing data streams. The communication relay process module includes means for selecting a UDP port to send and receive multiplexed data if multiplexable data streams bound for the same destination data gateway are present in the plurality of data streams.
The UDP port is selected from one of the data streams containing the data to be multiplexed. The communication relay process module also includes means for s sending the multiplexed data using the selected UDP port. The data gateways sends and receives data using a single UDP port for each remote gateway regardless of whether the data is multiplexed or not.
The present invention also provides a data gateway used in a communication system having a plurality of gateways for transferring data. Each gateway includes means for acquiring addresses wherein: data relay requests and information indicating relay destinations are received from other gateways; this relay destination address is used to find the closest remote gateway; and the address of the found destination gateway is acquired. Means for establishing connection establishes connections using this destination gateway. Means for is relaying generates packets from the data requested to be relayed and relays these packets to the destination gateway. This means for relaying to gateways includes an input buffer for receiving and temporarily storing the converted data. Means for acquiring time acquires time information. Means for acquiring multiplex timing determines whether multiplex timing has been reached based on the time acquired through means for acquiring time. When means for acquiring multiplex timing determines that this multiplex timing has been reached, means for performing multiplexing retrieves all the data stored in the input buffer and multiplexes streams having the same destination gateway into the same packets. This means for performing multiplexing includes: means for sending the multiplexed packets over the Internet; means for receiving packets from the Internet; means for disassembling the data received by the receiving means into separate streams if the data contains multiplexed packets; and an output buffer for temporarily holding the streams data disassembled by the means for disassembling.
The data gateway according to the present invention also includes interfaces connected to a plurality of communication lines. From the multiple streams received via these interfaces, a retriever uses the information specifying the destinations of these streams to determine destination gateways. A communication relay process module multiplexes parts of packets from streams determined by the retriever to have the same destination gateway. The partially multiplexed packets are sent to the destination gateway.
The communication relay process module of the data gateway according to the present invention includes a buffer for temporarily storing incoming data and timing acquisition means acquiring time information used as a basis for multiplex timing. Data is read from the buffer based on this timing acquisition means to perform multiplexing.
The present invention also provides a data relay system including a data gateway and a control unit. The control unit includes: means for acquiring data entered by a user indicating the number of data gateways connected to the data relay system to be administered; means for creating IP packets from commands containing the number of data gateways acquired through this acquiring means; and means for sending the number of data gateways, formed as packets by this means for creating IP packets, to each data gateway.
The data gateways include: means for receiving IP packets from the control unit indicating the number of data gateways; means for analyzing IP packet data received from this receiving means; means for reserving a port for sending and receiving multiplex data for each audio data gateway if the IP packet analyzed by this analyzing means contains the number of audio data gateways; means for saving at least the ports used for sending and receiving multiplexed data in association with the destination data gateways these ports are for; means for acquiring a multiplex UDP port corresponding to a relay destination data gateway if a new relay request is generated and the stream is determined to be a multiplexed stream as a result of multiplex negotiation with the destination data gateway; means for sending and receiving data using the multiplex port set up for each relay destination data gateway as obtained from this means for acquiring a multiplex UDP port.
The data relay program used by the data relay system is stored and distributed in a storage medium.